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Why GAO Did This Study 

The Department of Homeland 
Security (DHS) fiscal year 2006 
appropriations act provided $40.15 
million for the Immigration and 
Customs Enforcement's (ICE) 
program to modernize its 
information technology (IT) 
infrastructure. As mandated by the 
appropriations act, the department 
is to develop and submit for 
approval an expenditure plan for 
the program, referred to as "Atlas," 
that satisfies certain legislative 
conditions, including a review by 
GAO. In performing its review of 
the Atlas plan, GAO (1) determined 
whether the plan satisfies certain 
legislative conditions and 
(2) provided other observations 
about the plan and management of 
the program. To do this, GAO 
analyzed the fiscal year 2006 Atlas 
expenditure plan and supporting 
documents against the legislative 
conditions, federal requirements, 
and related best practices. GAO 
also interviewed relevant DHS 
officials. 



What GAO Recommends 



GAO is recommending that DHS 
minimize Atlas program risks by 
fully adhering to requirements 
development and management 
practices, implementing key 
contract management and 
oversight practices, completing 
planned risk management 
activities, and implementing critical 
performance management 
practices. DHS concurred with 
GAO's recommendations and 
described actions planned and 
under way to implement them. 

www.gao.gov/cgi-bin/getrpt7GAO-07-565. 

To view the full product, including the scope 
and methodology, click on the link above. 
For more information, contact Randolph C. 
Hite at (202) 512-3439 or hiter@gao.gov. 



What GAO Found 

The fiscal year 2006 Atlas expenditure plan, in combination with related 
program documentation and program officials' statements, satisfies or 
partially satisfies the legislative conditions set forth by Congress (see table). 

DHS's Satisfaction of Legislative Conditions 
Legislative conditions 

1 . Expenditure plan must meet the capital planning and investment control 
review requirements established by the Office of Management and Budget 
(OMB). 



Status 

partially satisfies 



2. DHS must ensure Atlas is compliant with DHS's enterprise 
architecture. 



partially satisfies 



3. The plan must comply with the acquisition rules, requirements, partially satisfies 
guidelines, and systems acquisition management practices of the federal 

government. 

4. The plan must include a certification by DHS's chief information officer partially satisfies 
that an independent verification and validation agent is currently under 

contract for the project. 

5. The plan must be reviewed and approved by the DHS Investment satisfies 

Review Board, the Secretary of Homeland Security, and OMB. 

Source: GAO. 

This satisfaction, however, is based on plans and commitments that provide 
for meeting these conditions rather than on completed actions to satisfy 
them. For example, to address the legislative condition related to capital 
planning and investment control review requirements, the program plans to, 
among other things, update its cost-benefit analysis in September 2007 to 
reflect emerging requirements and other program changes and to complete a 
privacy impact assessment by April 2007. In addition, the program is in the 
process of defining how it plans to use its independent verification and 
validation agent. 

GAO also observed that DHS has not implemented key system management 
practices. Specifically, 

• rigorous practices are not being fully adhered to in developing and 
managing system requirements, 

• key contract management and oversight controls have not been fully 
implemented, 

• planned risk management practices have yet to be implemented, and 

• performance management practices that are critical to measuring 
progress against program goals are still being implemented. 

Thus, much still needs to be accomplished to minimize the risks associated 
with the program's capacity to deliver promised IT infrastructure capabilities 
and benefits on time and within budget. Given that hundreds of millions of 
dollars are to be invested and the program is critical to supporting the ICE 
mission, it is essential that DHS follow through on its commitments to build 
the capability to effectively manage the program. Proceeding without it 
introduces unnecessary risks to the program and potentially jeopardizes its 
viability for future investment. 
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Abbreviations 


CCE 


Common Computing Environment 


CIO 


chief information officer 


CMMI 


Capability Maturity Model Integrated 


COTR 


contracting officer's technical representative 


DHS 


Department of Homeland Security 


EA 


enterprise architecture 


ICE 


Immigration and Customs Enforcement 


IT 


information technology 


IV&V 


independent verification and validation 


OMB 


Office of Management and Budget 


SACMM 


Software Acquisition Capability Maturity Model 


SEI 


Software Engineering Institute 
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permission from GAO. However, because this work may contain copyrighted images or 
other material, permission from the copyright holder may be necessary if you wish to 
reproduce this material separately. 
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United States Government Accountability Office 
Washington, D.C. 20548 



April 27, 2007 

The Honorable Robert C. Byrd 
Chairman 

The Honorable Thad Cochran 
Ranking Member 

Subcommittee on Homeland Security 
Committee on Appropriations 
United States Senate 

The Honorable David E. Price 
Chairman 

The Honorable Harold Rogers 
Ranking Member 

Subcommittee on Homeland Security 
Committee on Appropriations 
House of Representatives 

The 2006 Department of Homeland Security (DHS) appropriations act 1 
provided $40.15 million for Immigration and Customs Enforcement's (ICE) 
program to modernize its information technology (IT) infrastructure. The 
goals of the program — which consists of seven related IT projects and is 
referred to by ICE as the "Atlas" program — include improving information 
sharing and collaboration, strengthening information security, and 
improving workforce productivity. The act provides that none of the funds 
may be obligated until the department develops a plan for how the funds 
are to be spent that satisfies certain legislative conditions. The conditions 
included, among other things, having GAO review the plan. On November 9, 
2006, DHS submitted its fiscal year 2006 expenditure plan to the Senate and 
House Appropriations Subcommittees on Homeland Security. Pursuant to 
the act, we reviewed the plan. Our objectives were to (1) determine 
whether the plan satisfies legislative conditions specified in the act and 
(2) provide other observations about the expenditure plan and the 
department's management of the Atlas program. 

On February 2, 2007, we provided our review results via a briefing to the 
staffs of the Senate and House Appropriations Subcommittees on 
Homeland Security. This report summarizes our findings, conclusions, and 



'Pub. L. No. 109-90, Oct. 18, 2005. 
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recommendations from that briefing. The full briefing, including our scope 
and methodology, is reprinted in appendix I. 



DHS's fiscal year 2006 Atlas expenditure plan, including related program 
documentation and program officials' stated commitments, satisfies or 
partially satisfies key aspects of (1) meeting the capital planning and 
investment control review requirements of the Office of Management and 
Budget (OMB); 2 (2) complying with the DHS enterprise architecture; 3 
(3) complying with acquisition rules, requirements, guidelines, and systems 
acquisition management practices of the federal government; (4) including 
certification by the department's chief information officer (CIO) that an 
independent verification and validation agent is currently under contract 
for the project; and (5) having the plan reviewed and approved by the 
department's Investment Review Board, the Secretary of Homeland 
Security, and OMB. 

Regarding the fourth legislative condition, the department certified that an 
independent verification and validation agent is under contract for Atlas, 
and the department is in the process of defining how it plans to use its 
agent, but unresolved issues remain. For example, we have previously 
reported 4 that the independence of a verification and validation agent is 
critical to providing management with objective insight into program 
processes and associated work products. We also reported 5 that to be 
effective, the verification and validation function should be performed by 
an entity that is independent of the processes and products that are being 
reviewed. When the agent engaged to provide the service is a contractor, a 
practice to help ensure independence is to include in the agent's contract a 
provision that prohibits the agent from soliciting, proposing, or being 



2 OMB Circular A-ll, part 7, establishes policy for planning, budgeting, acquiring, and 
managing federal capital assets. 

3 An enterprise architecture provides a clear and comprehensive picture of an organization's 
operations and its supporting systems and infrastructure. It is an essential tool for 
effectively engineering business processes and for implementing and evolving supporting 
systems in a manner that maximizes interoperability, minimizes overlap and duplication, 
and optimizes performance. 

4 GAO, Homeland Security: Recommendations to Improve Management of Key Border 
Security Program Need to Be Implemented, GAO-06-296 (Washington, D.C.: Feb. 14, 2006). 

5 GAO-06-296. 
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awarded program work other than the independent verification and 
validation services and products. However, the contract that Atlas awarded 
to its agent did not include such a provision. 




Our observations address program efforts regarding (1) requirements 
development and management, (2) contract management and oversight, 
(3) risk management, and (4) performance management. An overview of 



• Rigorous practices are not being fully adhered to in developing and 
managing system requirements. On its three key projects, 6 the 
program has not fully implemented federal IT guidance, relevant best 
practices, and agency policies and procedures (i.e., ICE System 
Lifecycle Management Handbook) that provide for effective 
requirements development and management activities. For example, on 
its Common Computing Environment project, while the project 
established a policy to guide project efforts in developing and managing 
requirements, it did not among other things: 

• document stakeholder comments as part of the requirements 
elicitation process and review and obtain stakeholder approval on 
requirements until approximately 1 year after deploying system 
capabilities; 

• follow a disciplined change management process for several years 
(between 2003 and 2006) after the project was initiated in March 2003 
(during this time, requirements were introduced and implemented in 
an ad hoc fashion); and 

• develop supporting analysis showing traceability among 
requirements, systems design, and the contract. 

• Key contract management and oversight practices are not fully 
implemented. The program has not fully implemented practices that 
(1) the Software Engineering Institute's CMMI® 7 has identified as 



6 These three key projects are Common Computing Environment, Integration, and ICE 
Mission Information. 

7 CMMI is registered in the U.S. Patent and Trademark Office by Carnegie Mellon University. 
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essential to effectively managing and overseeing contracts that support 
IT projects and (2) the program has largely established in its existing 
policies and procedures. Specifically, the program has implemented two 
key practices, but has not fully implemented four others. For example, 
the program has assigned responsibility and authority for performing 
contract management and oversight, and has trained staff performing 
contract management and oversight. However, it has not, among other 
things, fully documented each contract — including clearly identifying 
the work to be performed and acceptance criteria — and verified and 
accepted deliverables. 

• Key risk management practices have yet to be implemented. We 
previously reported 8 that the program had developed a risk management 
plan and process to guide program office staff in managing risks 
throughout each project's life cycle and had recently begun to 
implement the risk management process. Since then, the program has 
taken additional steps to implement risk management, such as hiring a 
risk management coordinator and conducting quarterly program 
management meetings to review the program's progress in managing 
risks. However, the program has not fully implemented key practices 
essential to effectively managing project risks. For example, it does not 
have a transparent, documented, and traceable process for inventorying 
and resolving risks. Further, while the risk management plan defines a 
process and associated practices for developing risk mitigation 
strategies, the program has not fully implemented them. In addition, 
although the risk management plan calls for developing a process for 
elevating risks to management's attention, the program has not 
developed and implemented such a process. 

• Performance management practices are still being implemented and 
available data show mixed results and may not be reliable. 

We reported 9 in July 2006 that Atlas had begun taking steps to 
implement performance goals and measures. Since then, the program 
has taken additional steps to define and implement them. For example, 
the program identified 13 goals — for 5 of its 7 projects — that it has been 
using to measure progress during fiscal year 2006. However, it has yet to 



8 GAO, Information Technology: Immigration and Customs Enforcement Is Beginning to 
Address Infrastructure Modernization Program Weaknesses but Key Improvements Still 
Needed, GAO-06-823 (Washington, D.C.: July 27, 2006). 

9 GAO-06-823. 
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fully implement and institutionalize its performance goals and 
measures. For example, the program has not yet developed 
performance goals for 2 projects. 

In addition, progress against performance goals, as reported by the 
program, show mixed results. For example, of the 12 goals, the program 
has met 3 of them. Further, underlying data used by the program to 
measure performance may not be reliable in all cases. For example, on 1 
goal, the program uses project cost and schedule data from the 2005 Atlas 
business case in measuring progress against the goal, even though these 
data are out of date (e.g., they do not reflect project cost and schedule 
slippages that have occurred since 2005). 



Conclusions ^ ne fi sca l y ear 2006 Atlas expenditure plan, in combination with related 

program documentation and program officials' statements, satisfies or 
partially satisfies the legislative conditions set forth by Congress. However, 
this satisfaction is based on plans and commitments that provide for 
meeting these conditions, rather than on completed actions to satisfy the 
conditions. In addition, while steps are being initiated that are intended to 
address significant program management weaknesses, a number of 
improvements, including those recommended in our past reports, have yet 
to be implemented. 

Further, the program has not fully achieved many performance goals that it 
set out to accomplish over the past year. The above factors continue to put 
the program at risk and call for heightened and sustained management 
attention to expeditiously address and resolve the issues. Thus, there is 
much that still needs to be accomplished to minimize the risks associated 
with the program's capacity to deliver promised IT infrastructure 
capabilities and benefits on time and within budget. This includes 
demonstrating better progress against established performance goals in the 
coming year. 

Given that hundreds of millions of dollars are to be invested and the 
program is critical to supporting the ICE mission, it is essential that DHS 
follow through on its commitments to build the capacity to effectively 
manage the program. Proceeding without this capacity introduces 
unnecessary risks to the program and potentially jeopardizes its viability 
for future investment. 
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Recommendations for 
Executive Action 



To minimize risks to the Atlas program, we recommend that the Secretary 
of Homeland Security direct the Assistant Secretary for Immigration and 
Customs Enforcement to ensure that ICE follows through on its 
commitments to implement effective management controls and capabilities 
by implementing the following five recommendations: 

• Employ practices essential to ensuring that the Atlas program's 
independent verification and validation agent is and remains 
independent, including incorporating requirements in future contracting 
actions, such as the renegotiation or recompetition of the current 
independent verification and validation contract, which will prohibit the 
agent from soliciting, proposing, or being awarded program work other 
than providing independent verification and validation services and 
products. 

• Fully adhere to requirements development and management practices, 
including those specified in ICE's policies and procedures. This should 
also include having the Atlas program manager develop a process 
improvement plan for all of the projects that is consistent with the ICE 
System Lifecycle Management Handbook and provide for making the 
program manager and project managers responsible and accountable 
for rigorously adhering to the requirements in the handbook. 

• Fully implement key contract management and oversight practices, 
including those specified in ICE's policies and procedures. This should 
also include ensuring that the Atlas program manager, working with the 
program's contracting officer's technical representative, follow through 
on planned efforts to strengthen the program's compliance with these 
practices by (1) revising the acquisition plan by May 2007; (2) revising 
the ICE System Lifecycle Management Handbook by June 2007 to 
incorporate key contract management activities such as contract 
tracking and oversight; and (3) preparing statements of work (beginning 
in February 2007) for future contracts to clearly define Atlas work 
statements and acceptance criteria. 

• Complete implementation of planned risk management activities. This 
should include (1) updating the risk inventory to include risks for all 
projects and risk areas and (2) fully implementing and institutionalizing 
procedures for reporting to management on the existence and status of 
risks and progress in implementing mitigation strategies. 
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• Improve program performance management, including developing 
performance goals for projects that do not have goals and reporting on 
their progress in the fiscal year 2007 expenditure plan. Further, the 
program should assess that the data being used to measure progress are 
reliable, complete, and accurate. 



Agency Comments and 
Our Evaluation 



In DHS's written comments on a draft of this report, signed by the Director, 
Departmental GAO/Office of Inspector General Liaison Office, the 
department concurred with all of our recommendations and described 
actions planned and under way to implement them. The department also 
orally provided technical comments updating Atlas obligation amounts and 
the milestone for revising the acquisition plan. We incorporated these 
comments in the report and briefing as appropriate. DHS's written 
comments are reprinted in appendix II. 



We are sending copies of this report to the Chairman and Ranking Members 
of other Senate and House committees and subcommittees that have 
authorization and oversight responsibilities for homeland security. We are 
also sending copies to the Secretary of Homeland Security, the Assistant 
Secretary for Immigration and Customs Enforcement, and the Director of 
the Office of Management and Budget. Copies of this report will also be 
available at no charge on our Web site at www.gao.gov. 

Should you or your offices have any questions on matters discussed in this 
report, please contact me at (202) 512-3439 or at hiter@gao.gov. Contact 
points for our Offices of Congressional Relations and Public Affairs may be 
found on the last page of this report. Key contributors to this report are 
listed in appendix III. 




Randolph C. Hite 

Director, Information Technology Architecture 
and Systems Issues 
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Appendix I 



Briefing to the Staffs of the Subcommittees on 
Homeland Security, Senate and House 
Committees on Appropriations 



iGAO 

_ _ ™ Arrnuntability • Integrity • Reliability 

Information Technology: Immigration and Customs 
Enforcement Needs to Fully Address Significant Infrastructure 
Modernization Program Management Weaknesses 

Briefing to the staffs of the 

Subcommittees on Homeland Security 

Senate and House Committees on Appropriations 



February 2, 2007 



1 
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Appendix I 

Briefing to the Staffs of the Subcommittees 
on Homeland Security, Senate and House 
Committees on Appropriations 
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Outline of Briefing 
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Introduction 




Objectives, Scope, and Methodology 




Results in Brief 




Background 




Results 




■ Legislative Conditions 




■ Observations 




Conclusions 




Recommendations for Executive Action 




Agency Comments 




Attachment 1: Detailed Scope and Methodology 




Attachment II: GAO Analysis of Requirements Management 




2 
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Appendix I 

Briefing to the Staffs of the Subcommittees 
on Homeland Security, Senate and House 
Committees on Appropriations 
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The Department of Homeland Security's (DHS) Bureau of Immigration and Customs Enforcement 
(ICE) 1 is responsible for enforcing immigration, border security, trade, and other laws by, among other 
means, investigating and collecting intelligence on individuals and groups that act to violate these 
laws. ICE is also responsible for protecting federal facilities. 

Atlas is the ICE program developed to modernize the bureau's information technology (IT) 
infrastructure, which includes the hardware (e.g., servers, routers, storage devices, communication 
lines) and system software (e.g., database management and operating systems and network 
management) that provide an environment for operating and maintaining software applications. 

According to ICE, the goals of the Atlas program include improving information sharing, strengthening 
information security, and improving workforce productivity. 

The fiscal year 2006 Department of Homeland Security Appropriations Act 2 appropriated $40.15 
million for Atlas 3 and stated that none of the funds may be obligated until the department submits for 
approval to the Senate and House Committees on Appropriations a plan for how the funds are to be 
spent that satisfies the following legislative conditions: 



'ICE was formed from the former Immigration and Naturalization Service, U.S. Customs Service, and other entities. Atlas began in 2002 
under the former Immigration and Naturalization Service. 
2 Pub. L. No. 109-90, Oct. 18, 2005. 

3 ln the act, appropriations for Atlas are under the heading Automation Modernization for Immigration and Customs Enforcement. 

3 
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■ meets the capital planning and investment control review requirements established by the Office of 
Management and Budget (OMB), including OMB Circular A-1 1 , part 7; 4 

■ complies with DHS's information systems enterprise architecture; 

■ complies with the acquisition rules, requirements, guidelines, and systems acquisition management 
practices of the federal government; 

■ includes certification by the DHS chief information officer (CIO) that an independent verification and 
validation agent is currently under contract for the project; 

■ is reviewed and approved by DHS's Investment Review Board, 5 the Secretary of Homeland 
Security, and OMB; and 

■ is reviewed by GAO. 

DHS submitted its fiscal year 2006 expenditure plan for $39. 75 6 million on November 9, 2006, to the 
Senate and House Appropriations Subcommittees on Homeland Security. 



"OMB Circular A-1 1 establishes policy for planning, budgeting, acquisition, and management of federal capital assets. 
5 The purpose of the Investment Review Board is to integrate capital planning and investment control, budgeting, and acquisition. It is 
also to ensure that spending on investments directly supports and furthers the mission and that this spending provides optimal benefits 
and capabilities to stakeholders and customers. 

6 DHS's fiscal year 2006 appropriations (Pub. L. No. 109-90, Oct. 18, 2005) provided $40.15 million for Atlas. In December 2005, this 
amount was reduced to $39.75 million via a 1 percent governmentwide rescission in Pub. L. No. 109-148, Division B, Dec. 30, 2005. 
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Briefing to the Staffs of the Subcommittees 
on Homeland Security, Senate and House 
Committees on Appropriations 
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As agreed, our objectives were to 

■ determine whether the Atlas fiscal year 2006 expenditure plan satisfies the legislative conditions 
and 

■ provide other observations about the expenditure plan and DHS's management of the Atlas 
program. 

To address our first objective, we analyzed the fiscal year 2006 Atlas expenditure plan and supporting 
documents against legislative conditions and relevant federal requirements and related best practices 
to determine whether the conditions were being met. 

To address our second objective, we evaluated supporting documentation and interviewed relevant 
officials to determine progress in establishing program management capabilities in areas such as 
requirements development and management, contract management and oversight, risk management, 
and performance management. In assessing requirements management and contract management 
and oversight, we focused on three projects — Common Computing Environment, Integration, and ICE 
Mission Information — that collectively account for approximately 75 percent of the funds provided to 
the program and that the Atlas program manager identified as being the most critical to the program's 
success and ICE's mission. We conducted our work at DHS and ICE headquarters in Washington, 
D.C., from July 2006 through January 2007 in accordance with generally accepted government 
auditing standards. Details of our scope and methodology are provided in attachment 1 . 



5 
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Briefing to the Staffs of the Subcommittees 
on Homeland Security, Senate and House 
Committees on Appropriations 
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Results in Brief 



Objective 1: Satisfaction of Legislative Conditions 



Legislative conditions 


Status 


1 . Meets the capital planning and investment control review requirements established by OMB, 
including OMB Circular A-1 1 , part 7. 


partially satisfies 7 


2. Complies with the DHS information systems enterprise architecture. 


partially satisfies 


3. Complies with the acquisition rules, requirements, guidelines, and systems acquisition 
management practices of the federal government. 


partially satisfies 


4. Includes a certification by the chief information officer of the Department of Homeland Security 
that an independent verification and validation agent is currently under contract for the project. 


partially satisfies 


5. Is reviewed and approved by the DHS Investment Review Board, Secretary of Homeland 
Security, and OMB. 


satisfies 8 


6. Is reviewed by GAO. 


satisfies 



'Partially satisfies=the plan, in combination with supporting documentation and stated commitments by program officials, either satisfies 
or provides for satisfying many, but not all, key aspects of the condition that we reviewed. 

8 Satisfies=the plan, in combination with supporting documentation and stated commitments by program officials, either satisfies or 
provides for satisfying every aspect of the condition that we reviewed. 
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Objective 2: Other Observations 

DHS has not implemented key system management practices essential to establishing an effective program 
management control environment. Specifically, in managing the Atlas program, the department has not 

■ fully adhered to rigorous practices called for in developing and managing system requirements, 

■ implemented key contract management and oversight controls, 

■ completed implementation of planned risk management practices, or 

■ implemented performance management practices critical to measuring progress against program 
goals. 

These shortfalls are attributable in part to a number of factors, including the Atlas program's expedited 
schedule to deploy modernized IT infrastructure capabilities to users in the wake of the events of September 
1 1 , 2001 , and in the haste of doing so, not following these practices or related ICE policies and procedures. 
Until these practices are fully implemented, the program is at an increased risk of not being able to deliver 
modern IT infrastructure capabilities on time and within budget. Program officials acknowledge the shortfalls 
and have efforts planned or under way to address some, but not all, of the missing practices. 

Accordingly, we are making recommendations to the Secretary of Homeland Security to strengthen the Atlas 
program by implementing and institutionalizing these key system management practices, thereby helping to 
establish an effective program management control environment. In commenting on a draft of this briefing, 
ICE officials, including the deputy chief information officer who is also the Atlas program manager, agreed 
with our results. 



7 
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ICE was formed as a component agency of DHS in 2003 when the law enforcement functions of the 
Department of Justice's former Immigration and Naturalization Service and Department of the 
Treasury's former Customs Service and other agencies were merged into DHS. 

The ICE mission is to ensure the security of the American people and homeland by, among other 
means, investigating violators of and enforcing the nation's immigration and customs laws; policing 
and securing federal buildings and other facilities; and collecting, analyzing, and disseminating 
intelligence to assist in these endeavors. Headed by the Assistant Secretary of Immigration and 
Customs Enforcement, ICE has approximately 15,000 employees in more than 400 offices in the 
United States and in other countries. 



The Atlas program was started by the Immigration and Naturalization Service in 2002. Responsibility 
for the program was transferred to ICE in 2003. The figure on the following slide is a simplified version 
of the current ICE and Atlas organizational placements within DHS. 
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Background 

ICE and Atlas Organizational Placement 



Department of 
Homeland Security 
Secretary 

Deputy Secretary 



Under Secretary for 
Management 



Chief Information Officer 



Chief Financial Officer 



Under Secretary for 
Science and Technology 



Under Secretary for 
Preparedness 




Federal 
Emergency 
Management 
Agency 



Office of 




Office of 


Investigations 




Intelligence 



Source: GAO analysis of DHS data. 
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Background 

Program Impetus and Goals 
Impetus for Program 

According to ICE officials, Atlas was initiated to address information sharing and security limitations 
within the former Immigration and Naturalization Service caused by, for example, 

■ obsolete hardware/software; 

■ incompatible, noninteroperable information systems; and 

■ system security weaknesses. 

These officials stated that these challenges were exacerbated by the formation of ICE because the 
organizations merged into ICE had different hardware/software environments (e.g., multiple e-mail 
systems) and different missions and customer needs. 

Program Goals 

The stated goals of Atlas are, among other things, to 

■ promote information sharing and collaboration, 

■ strengthen information security, and 

■ enhance workforce productivity. 
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Background 

Atlas Projects 


Atlas Consists of Seven Projects 


Project 


Description 


Common Computing 
Environment 


■ Deploy compatible desktops, servers, and standard office automation applications. 

■ Deploy a common e-mail platform. 

■ Integrate ICE with a shared departmental active directory structure. 


Integration 


■ Migrate from the ICE network to the DHS OneNetwork infrastructure. 

■ Provide streaming video. 

■ Refresh local area network cabling and electronics and replace legacy firewalls. 

■ Refresh the wide area network and local area network communications hardware at the overseas ICE 
attache offices. 

■ Provide network support for all applications and provide converged communications among all system 
users. 


ICE Mission Information 


■ Provide enterprise data warehousing capabilities and integrate related ICE business processes. 

■ Organize information and support user access so ICE users can find relevant, timely, and reliable 
information. 

■ Improve information searching and indexing capabilities and integrate legacy applications with service- 
oriented techniques. 


Information Assurance 


■ Institute strong information, system/application platform, network, and computer security measures. 


Architecture Engineering 


■ Provide state-of-the-art engineering facilities and tools to ensure new technologies align to the DHS 
enterprise architecture (EA). 


Data Center Migration 


■ Migrate ICE hardware and applications from the Department of Justice data centers to the common 
DHS data center. 

■ Ensure compliance with the DHS EA. 


Transformation Planning 


■ Oversee and coordinate the various Atlas projects. 

■ Ensure compliance with the DHS EA. 

■ Ensure adherence to project cost, schedule, and performance goals. 



Source: GAO analysis of DHS data. 
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Background 

Atlas Appropriations 



Appropriated Funds 



Millions 
of dollars 



160 
140 
120 
100 
80 
60 
40 



Since fiscal year 2002, 
Atlas has been provided 
$197. 9 million 

in appropriated funds 



S39.1 million 9 

from INS 
counterterrorism 
funding in 
fiscal year 2002 



$24.5 million 

from INS investment 
and operations and 
maintenance 
funding in 
fiscal year 2003 



$40.0 million 15 

from DHS 
appropriations in 
fiscal year 2004, of 
which $30.2 million 
was reprogrammed 
by DHS to cover 
other ICE priorities, 
leaving $9.8 million 



$15.0 million e 
$39.7 million d from DHS 
from DHS appropriations in 

. appropriations, in fiscal year 2007 

fiscal year 2006 




FY03 ' FY04 ' FY05 

Source: GAO analysis based on DHS appropriation laws. 



Counterterrorism funding came from Pub. L. No. 107-117, Jan. 10, 2002; H.R. Rep. 107-350, Dec. 19, 2001. 
"Pub. L. No. 108-90, Oct. 1, 2003. 
°Pub. L. No. 108-334, Oct. 18, 2004. 

"DHS's fiscal year 2006 appropriations (Pub. L. No. 109-90, Oct. 18, 2005) provided $40.15 million for Atlas. In December 2005, this 
amount was reduced to $39.7 million via a 1 percent governmentwide rescission in Pub. L. No. 109-148, Division B, Dec. 30, 2005. 
e Pub. L. No. 109-295, Oct. 4, 2006. The law mandates that DHS also submit an expenditure plan prior to obligation of these funds. 
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Background 

Atlas Obligations 



ICE reports that it has obligated $84.7 million of the $1 13.0 million available for Atlas in fiscal years 
2002, 2003, 2004, and 2005. 



Atlas funds obligated (by project) for fiscal years 2002-2005 
(as of December 31, 2006) 


Project 

(in order presented in the plan) 


Obligations 
(in millions) 


Total 


FY 2002 


FY 2003 


FY 2004 


FY 2005 


Common Computing Environment 


$0 


$10.5 


$3.2 


$4.3 


$18.0 


Integration 


6.1 


20.4 


2.5 


0.5 


29.5 


ICE Mission Information 


1.3 


4.6 


0.8 


4.0 


10.7 


Information Assurance 


10.2 


9.6 





0.6 


20.4 


Architecture Engineering 











0.7 


0.7 


Data Center Migration 

















Transformation Planning 





0.9 


3.3 


1.2 


5.4 


TOTAL 


$17.6 


$46.0 


$9.8 


$11.3 


$84.7 



Source: GAO analysis of DHS data. 
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Background 

Planned Use of Fiscal Year 2006 Funding 

According to the fiscal year 2006 expenditure plan, the $39.75 million is to be spent on the projects as 
follows. 



Distribution of funding for Atlas projects (in order presented in the plan) 


Project 


Funding 
(in millions) 


Planned use 


Common Computing 
Environment 


$23.88 


■ Complete acquisition of hardware refresh 

■ Deploy active directory and e-mail exchange system 


Integration 


2.30 


■ Replace outdated communications electronics at foreign attaches to ensure 
compatibility with current ICE Net WAN and LAN infrastructure standards 

■ Complete acquisition of streaming video for field sites 


ICE Mission Information 


5.84 


■ Complete acquisition of technology to efficiently integrate all related ICE business 
processes 

■ Complete 60 percent of acquisition of technology to support data warehousing 
capabilities 


Information Assurance 


0.92 


■ Acquire IT security specialists to design and develop single sign-on and audit log 
capabilities 


Architecture 
Engineering 





■ No use of fiscal year 2006 funds identified 


Data Center Migration 


0.89 


■ Plan for the migration of ICE hardware and applications from two Department of 
Justice data centers to a common DHS data center 


Transformation 
Planning 


5.92 


■ Provide $2.3 million to continue performing and planning program management 
activities, including maintaining the Atlas business case 

■ Provide $3.6 million in management reserve for unanticipated program costs or to 
fund priority emerging IT modernization requirements 


TOTAL 


$39.75 





Source: GAO analysis of DHS data. 
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Objective 1 Results 

Legislative Conditions 

The Atlas expenditure plan satisfies or partially satisfies each of the legislative conditions. 

Condition 1 partially satisfied. The expenditure plan, including related program documentation and 
statements from the program manager, partially satisfies the capital planning and investment control 
review requirements established by OMB, including Circular A-1 1 , part 7, which establishes policy for 
planning, budgeting, acquisition, and management of federal capital assets. 

Examples of our analysis are included in the following table. As the table shows, not all OMB 
requirements have been satisfied, but oral commitments have been made for doing so. Given that 
ICE has reportedly already invested $75.4 million on Atlas projects and plans to invest another $39.7 
million this year, it is important for ICE to follow through on these commitments. 
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Objective 1 Results 

Legislative Conditions 


GAO Analysis of Atlas Compliance with Legislative Conditions 


FynmnlPQ of OMR Oirpular 

A-11 requirements 


Dpcultc of our anal\/<;ic; 
ncouiio \J i uui ci 1 i a i y o i o 


Indicate whether the investment 
was approved by an investment 
review committee 


The plan was approved on May 1 1 , 2006, by DHS's Deputy Secretary, who chairs the DHS 
Investment Review Board. 


Provide justification and describe 
acquisition strategy 


A business case, including a cost-benefit analysis, was issued in December 2005 to provide 
economic justification for the program. It estimated the program's life cycle costs to be 
approximately $1 billion through the year 2024. The program plans to issue an updated cost- 
benefit analysis in September 2007 to reflect emerging requirements and other program 
changes (e.g., cost and schedule slippages). In addition, the expenditure plan and supporting 
documents (e.g., fiscal year 2006 Atlas budget submission to OMB known as an exhibit 300) 
provide aspects of a high-level acquisition strategy, such as identifying the ICE contracts that 
are being used and are to be used to acquire hardware and software products and program 
support services. An acquisition plan was approved in February 2006 that includes a statement 
of need and the capabilities to be delivered through these contracts. To help ensure compliance 
with contract criteria and Atlas program objectives, a contracting officer's technical 
representative (COTR) was hired in November 2006. The program's efforts to ensure 
compliance with contract management and oversight practices are more fully discussed in the 
Observations section of this briefing. 


Summarize life cycle costs and 
cost-benefit analysis, including the 
return on investment 


The December 2005 cost-benefit analysis provides costs and benefits for the life cycle of Atlas, 
which is through the year 2024. The analysis also included an estimated return on investment 
for three alternative approaches. As we previously reported, 9 the analysis was in large part 
consistent with OMB and best practices guidance, but it did omit key practices such as key 
mission requirements. According to the program manager, the analysis is considered to be a 
"living document" and the program plans to issue an update in September 2007 that, among 
other things, addresses these requirements. 



8 GAO, Information Technology: Immigration and Customs Enforcement Is Beginning to Address Infrastructure Modernization Program 
Weaknesses but Key Improvements Still Needed, GAO-06-823 (Washington, D.C.: July 27, 2006). 
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Objective 1 Results 

Legislative Conditions 



GAO Analysis of Atlas Compliance with Legislative Conditions (cont.) 



Examples of OMB Circular 
A-11 requirements 


Results of our analysis 


ProvidG performance goals and 
measures 


The expenditure plan and supporting documentation (e.g., prior expenditure plans) identify 13 
proposed goals and measures, but two projects do not have measures. Proposed measures have 
been drafted for these projects, but target dates for their completion have not been identified by 
the program. In addition, as part of the December 2005 Atlas business case, the program 
mapped Atlas's goals to ICE's mission and goals. Our analysis of Atlas's performance 
management is discussed in more detail in the Observations section of this briefing. 


Address security and privacy 


The plan and supporting documentation state the importance of security and privacy and provide 
high-level information on intended security measures, including one proposed project — 
information assurance — that is intended to implement an ICE security program. The plan 
allocates $0.9 million to this project in addition to the $1 .9 million provided in the fiscal year 2005 
plan. In addition, the program issued an Atlas system security plan, as well as a security test and 
evaluation plan, in April 2006. The program also recently issued an updated system security plan 
and security test and evaluation plan in December 2006. ICE also developed a draft Atlas privacy 
impact assessment in August 2005, submitted a revised assessment in August 2006 to the DHS 
privacy office, and plans to complete and submit the final Atlas privacy impact assessment by 
April 2007. 


Provide for managing risk 


The program issued a risk management plan in January 2006. This plan provides guidance for 
identifying, analyzing, and resolving program risks before they occur. In addition, a risk 
management coordinator was hired in August 2006 to help identify and monitor risks. According 
to the program manager, this official recently vacated the position, and the program is currently in 
the process of hiring a new coordinator. (At our Feb. 1 , 2007, exit meeting with DHS and ICE 
officials, the program manager told us a coordinator had been hired on Jan. 22, 2007.) Further, 
while the program procured an automated tool in May 2006 to help complete the inventory and 
manage risks, it does not have, among other things, a complete inventory of all program risks. 
Our complete findings related to risk management are addressed in the Observations section of 
this briefing. 



Source: GAO analysis of DHS data. 
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Objective 1 Results 

Legislative Conditions 

Condition 2 partially satisfied. The plan, including related program documentation and DHS 
officials' statements, partially satisfies the condition that the department ensures Atlas is compliant 
with DHS's enterprise architecture (EA). 

An EA provides a clear and comprehensive picture of an organization's operations and its supporting 
systems and infrastructure. It is an essential tool for effectively and efficiently engineering business 
processes and for implementing and evolving supporting systems in a manner that maximizes 
interoperability, minimizes overlap and duplication, and optimizes performance. We have worked with 
Congress, OMB, and the federal Chief Information Officers Council to highlight the importance of 
architectures for both organizational transformation and IT management. An important element of EA 
management is ensuring that IT investments are compliant with the EA, including basing such 
assessments on documented analysis. 

The DHS Enterprise Architecture Board, supported by the Enterprise Architecture Center of 
Excellence, 10 is responsible for ensuring that projects demonstrate adequate technical and strategic 
compliance with the department's EA. To this end, the board conducts compliance reviews at key 
decision points in an investment's life cycle. Specifically, DHS guidance" directs the board prior to an 
investment's acquisition milestone (referred to by DHS as key decision point 2) to assess the 
investment against the transition strategy, data architecture, application component, and technology 
architecture. 
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10 A review group made up of subject matter experts that recommends EA compliance to the DHS Enterprise Architecture Board and 
ultimately to the DHS Investment Review Board. 

"Department of Homeland Security Enterprise Architecture Board: EAB Governance Process Guide, August 9, 2004, draft version 2.0. 
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Objective 1 Results 

Legislative Conditions 

We previously reported 12 in May 2005 that the Atlas program manager requested that the center 
assess the program's compliance with the EA. The center staff compared Atlas to version 2.0 of the 
DHS EA and reported the results of its assessment to the board, stating that Atlas was in compliance 
with the DHS EA. In July 2005, the board approved this compliance determination. 

Subsequently, we reported 13 that ICE's compliance determination was not based on a documented 
analysis mapping Atlas's infrastructure to the DHS EA. Specifically, the department did not have a 
documented methodology for evaluating programs for compliance with the DHS EA, other than 
relying on the expertise of the Center of Excellence members. 

Since the July 2005 compliance determination, DHS issued version 2006 of its EA. 14 In addition, Atlas 
has undertaken an effort to define an architecture framework and technical environment for a data 
warehouse aimed at providing reporting and analytical capabilities to investigative and enforcement 
personnel. Despite these changes, no new compliance reviews have been conducted. According to 
the Atlas program manager, a compliance review was not required during this time as the program 
had not yet reached a phase requiring it. The program manager stated the department has scheduled 
an EA compliance review in March 2007. 
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,2 GAO-06-823. 
,3 GAO-06-823. 

"On August 14, 2006, we reported that version 2006 of DHS's EA was at stage 2 of the Enterprise Architecture Management Maturity 
Framework, demonstrating that DHS has established the foundational commitments and capabilities needed to manage the 
development of the architecture. See GAO, Enterprise Architecture: Leadership Remains Key to Establishing and Leveraging 
Architectures for Organizational Transformation, GAO-06-831 (Washington, D.C.: Aug. 14, 2006). 
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Objective 1 Results 

Legislative Conditions 

In addition, program officials told us that they are taking other steps aimed at ensuring compliance. 
These steps include 

■ participating in the DHS Enterprise Architecture Center of Excellence and the Data Management 
Working Group; these groups are currently tasked with developing the DHS EA and the DHS Data 
Reference Model, 

■ participating in DHS EA initiatives, such as serving on the Service Oriented Architecture Tactical 
Working Group, 15 which is to provide the framework for departmentwide service alignment and 
interoperability efforts, and 

■ establishing an Atlas Interoperability Lab — under the Architecture Engineering project — to among 
other things, help ensure the program's projects are compliant with the EA. 

These steps notwithstanding, until DHS demonstrates, through verifiable documentation and a 
methodologically based analysis, that ICE is aligned with the DHS enterprise architecture, the 
program will remain at risk of being defined and implemented in a way that does not support optimal 
departmentwide operations, performance, and achievement of strategic goals and outcomes. 
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,5 The Tactical Working Group is responsible for developing strategies to address service oriented architecture design and operational 
issues. 
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Objective 1 Results 

Legislative Conditions 

Condition 3 partially satisfied. The plan, including related program documentation and statements 
from the Atlas program manager, partially satisfies the condition to comply with the acquisition rules, 
requirements, guidelines, and systems acquisition management practices of the federal government. 
These practices provide a management framework based on the use of rigorous and disciplined 
processes for planning, managing, and controlling the acquisition of IT resources, including: 

■ acquisition planning, which ensures, among other things, that reasonable plans, milestones, and 
schedules are developed and that all aspects of the acquisition effort are included in these plans; 

■ procurement, which involves making sure that (1) a request for proposals delineating a project's 
requirements is prepared and (2) consistent with relevant acquisition laws and regulations, a 
contractor is selected that can cost-effectively satisfy these requirements; 

■ requirements development and management, which includes establishing and maintaining a 
common and unambiguous definition of requirements among the acquisition team, the system 
users, and the development contractor; 

■ project management, which provides for management of the activities within the project office and 
supporting contractors to ensure a timely, efficient, and cost-effective acquisition; 

■ contract tracking and oversight, which ensures that the development contractor performs according 
to the terms of the contract; needed contract changes are identified, negotiated, and incorporated 
into the contract; and contractor performance issues are identified early, when they are easier to 
address; and 



i 

iGAO 

Accountability • Integrity • Reliability 



21 



Page 28 



GAO-07-565 Information Technology 



Appendix I 

Briefing to the Staffs of the Subcommittees 
on Homeland Security, Senate and House 
Committees on Appropriations 



Objective 1 Results 

Legislative Conditions 

■ evaluation, which determines whether the acquired products and services satisfy contract 
requirements before acceptance. 

These acquisition management processes are also embodied in published best practices models, 
such as the Software Engineering Institute's Software Acquisition Capability Maturity Model®. 16 

Examples of our analysis of ICE performance of these processes and practices are shown in the 
following table. They show that not all aspects of the processes and practices have been 
implemented, but that oral commitments have been made for doing so. 
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16 Developed by Carnegie Mellon Software Engineering Institute (SEI), Software Acquisition Capability Maturity Model (SA-CMM* ) 
version 1 .03 (March 2002). 
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Objective 1 Results 

Legislative Conditions 



GAO Analysis of Atlas Compliance with Legislative Conditions 



Example of practices 


Results of our analysis 


Acquisition Planning 

Ensures that reasonable plans, 
milestones, and schedules are 
developed and that all aspects of 
the acquisition effort are included 
in these plans. 


The expenditure plan and supporting documents (e.g., fiscal year 2006 Atlas budget submission 
to OMB known as an exhibit 300) provide aspects of a high-level acquisition strategy, such as 
identifying the ICE contracts that are being used and are to be used to acquire products and 
program support services. An acquisition plan was issued in February 2006, and it includes a 
statement of the capabilities to be delivered through these contracts. 

To manage and oversee the contracts, including helping ensure compliance with contract 
performance criteria and Atlas program objectives, the program hired a COTR in November 
2006. In addition, the business case issued in December 2005 provides details on alternatives, 
cost, and schedule. The program plans to issue an updated business case and cost-benefit 
analysis by September 2007 to reflect emerging requirements and other program changes (e.g., 
project cost and schedule slippages). In addition, Atlas is also updating and revising its January 
2006 acquisition program baseline, which is to include cost and schedule baselines for its 
projects, with the goal of finalizing it by March 2007. 

According to the program manager, Atlas is currently following the February 2005 ICE System 
Lifecycle Management Handbook.' 7 The handbook addresses a number of key process areas 
such as project management and requirements development and management; however, it does 
not address certain key acquisition management activities such as solicitation and contract 
tracking and oversight. According to the Atlas program manager, he acknowledges this problem 
and plans to update the handbook by June 2007 to include the missing acquisition management 
activities. This area is more fully discussed in the Observations section of this briefing. 
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,7 U.S. Immigration and Customs Enforcement: System Lifecycle Management Handbook, The Systems Development Lifecycle of 
Immigration and Customs Enforcement, February 2005, version 1 .0. 
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Objective 1 Results 

Legislative Conditions 



GAP Analysis of Atlas Compliance with Legislative Conditions (cont.) 



Example of practices 


Results of our analysis 


Project Management 

Provides for the management of 
activities within the project office 
and supporting contractors to 
ensure a timely, efficient, and cost- 
effective acquisition. 


ICE has established a program office with responsibility for managing the acquisition, 
deployment, operation, and sustainment of Atlas. The Atlas program management office is to be 
allocated funding of $2.3 million via the Transformation Planning project in the expenditure plan. 
The current staffing of the program office consists of a program manager, who is also the deputy 
chief information officer; a deputy program manager; a manager for each of the seven Atlas 
projects; and other personnel (e.g., a business analyst or a communications specialist). 
According to the program manager, the program office is now fully operational. In addition, a 
project plan for each of the projects was approved by the program manager in the latter part of 
2006. 



Source: GAO analysis of DHS data. 
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Objective 1 Results 

Legislative Conditions 

Condition 4 partially satisfied. DHS partially satisfies the legislative condition requiring that the plan 
include a certification by the department's CIO that an IV&V agent is currently under contract for the 
project. 

In an October 25, 2006, letter to ICE's CIO, DHS's Deputy CIO certified that an IV&V agent is under 
contract for Atlas. However, there are unresolved issues with the Atlas program's efforts in this area. 
First, in the Deputy ClO's letter, he requested additional information and made recommendations 
regarding the IV&V agent, asking for a response by January 31 , 2007. Specifically, this official 

■ requested a copy of the IV&V contract and the agent's work plan in order to validate, among other 
things, whether planned IV&V activities are in accordance with industry quality standards and 
whether the agent has sufficient staff and 

■ recommended that the Atlas program (1) review whether the information assurance and security 
experience of key IV&V personnel was sufficient and (2) appoint a full-time COTR before the end of 
the first quarter of 2007. 

The program manager stated that he is currently drafting the response on behalf of ICE's CIO with the 
goal of transmitting it to DHS by the first or second week of February 2007. 



i 

iGAO 

Accountability * Integrity * Reliability 



25 



Page 32 



GAO-07-565 Information Technology 



Appendix I 

Briefing to the Staffs of the Subcommittees 
on Homeland Security, Senate and House 
Committees on Appropriations 



Objective 1 Results 

Legislative Conditions 

Second, as we have previously reported, 18 the independence of an IV&V agent is critical to providing 
management with objective insight into program processes and associated work products. We also 
reported that to be effective, the verification and validation function is to be performed by an entity 
that is independent of the processes and products that are being reviewed. When the agent engaged 
to provide the service is a contractor, a practice to help ensure independence is to include in the 
agent's contract a provision that prohibits the agent from soliciting, proposing, or being awarded 
program work other than the IV&V services and products. 

The contract that Atlas awarded to its IV&V agent (dated Sept. 13, 2006) does not include such a 
provision. Instead, the contract states that the contractor will notify the agency of any potential 
conflicts of interest in performing work under the contract. The Atlas program manager told us he did 
not believe incorporating the contractual provision called for by the key practice was necessary 
because the agent's only line of work is performing IV&V activities, and he agreed to provide 
documentation to support this. He also said that in the event the agent solicited, proposed, or is 
awarded other work on the Atlas program, he would replace the agent with one that was independent. 
However, the program manager did not provide documentation to IV&V being the agent's sole line of 
business. In addition, having to replace an agent could delay program activities and is not consistent 
with the key practice to avoid such problems. Consequently, until the practice to ensure 
independence is included in the agent's contract, there is an increased risk that the agent's 
independence could become impaired, resulting in the program being unable to rely on the agent's 
work. 
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18 GAO, Homeland Security: Recommendations to Improve Management of Key Border Security Program Need to Be Implemented, 
GAO-06-296 (Washington, D.C.: Feb. 14, 2006). 
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Objective 1 Results 

Legislative Conditions 

Condition 5 satisfied. DHS and OMB satisfied the legislative condition requiring that the plan be 
reviewed and approved by the DHS Investment Review Board, Secretary of Homeland Security, and 
OMB. 

• The DHS Deputy Secretary, who chairs the DHS Investment Review Board, approved the plan 
on May 1 1 , 2006. 

• OMB approved the plan on September 14, 2006. 

Condition 6 satisfied. GAO satisfied the condition that it review the plan. 
Our review was completed in January 2007. 
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Objective 2 Results 

Observation 1 : Requirements Management 

Observation 1: Atlas did not fully adhere to rigorous requirements development and 
management practices 

Federal IT management guidance and relevant best practices 19 recognize the importance of 
effectively developing and managing system requirements. According to this criteria, well-defined 
requirements are important because they establish agreement among the various stakeholders on 
what the system is to do, how well it is to do it, and how it is to interact with other systems. Effective 
requirements development and management involves, among other things: 

■ establishing a requirements management policy; 

■ eliciting desired or required system capabilities from users and translating them into system 
requirements and obtaining approval from all stakeholders before deploying system capabilities; 

■ establishing a system concept of operations, which describes the business process to be supported 
and how users will interact with the proposed system, and defines system requirements; 

■ ensuring that changes to requirements are controlled and approved via a disciplined change control 
process as the system is developed and implemented; and 

■ maintaining bidirectional traceability, meaning that a given requirement can be traced backward to 
systems documentation and forward to the appropriate contract vehicle and system components 
that will satisfy the requirement and the test that will verify that it is satisfied. 
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'See, for example, Software Engineering Institute, Capability Maturity Model Integrated (CMMI), version 1.1 (March 2002). 
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Objective 2 Results 

Observation 1 : Requirements Management 

As we reported in July 2006, 20 ICE incorporated these practices in its ICE System Lifecycle 
Management Handbook, 21 which was issued in February 2005. However, the program did not fully 
follow these practices on its three key projects: Common Computing Environment (CCE), Integration, 
and ICE Mission Information. For example, on CCE, while the project established the ICE System 
Lifecycle Management Handbook as the policy to guide project efforts in developing and managing 
requirements, it did not 

■ document stakeholder comments as part of the requirements elicitation process and review and 
obtain stakeholder approval on requirements until approximately 1 year after deploying system 
capabilities; 

■ develop a concept of operations until several years after the project had been initiated, specifically, 
the project was initiated in March 2003 and began deploying system capabilities in December 
2005, but the concept of operations was not developed until June 2006; 

■ follow a disciplined change management process for several years (between 2003 and 2006) after 
the project was initiated in March 2003 (during this time, requirements were introduced and 
implemented in an ad hoc fashion); and 

■ develop supporting analysis showing traceability between requirements, system design, and the 
contract. 
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2 °GAO-06-823. 

2, U.S. Immigration and Customs Enforcement: System Lifecycle Management, The Systems Development Lifecycle of Immigration and 
Customs Enforcement, version 1.0 (Washington, D.C.: February 2005). 
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Objective 2 Results 

Observation 1 : Requirements Management 

Our assessment of the extent to which the program complied with these practices on the three 
projects is shown in attachment II. 

According to ICE officials, including the program and project managers, the reason that the Atlas 
projects did not fully adhere to these practices is due to the agency's expedited schedule to deploy 
modernized IT infrastructure capabilities to users in the wake of the events of September 1 1 , 2001 . 
Specifically, the officials stated that in their haste to deliver capabilities, the project manager skipped 
certain system management life cycle practices such as these. 

The program manager acknowledged that requirements management needs to be improved and 
stated that key steps are being taken to ensure the projects closely follow the practices specified in 
the ICE System Lifecycle Management Handbook. He cited recent efforts by the CCE project 
manager to follow such practices on work recently initiated to refresh hardware, such as desktops 
and laptop computers across the agency. For example, in January 2007, the project conducted a 
workstation requirements analysis survey that was sent out to ICE users to identify requirements 
regarding the current workstation being used and issues that needed to be addressed in order to 
perform functions associated with their jobs. In accordance with the handbook, project staff drafted 
requirements, which they plan to finalize and approve by February 2007. While these are steps in the 
right direction on this particular project, the program manager was not able to provide us with a 
process improvement plan on how the program going forward would ensure compliance with 
requirements development and management practices across all of the projects. 
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Objective 2 Results 

Observation 1 : Requirements Management 

Consequently, until Atlas fully implements effective requirements development and management 
practices programwide, it faces the increased likelihood that its projects will not meet user needs, 
operate as intended, or be delivered on time and within budget. In addition, there is evidence that this 
has already occurred. For example, on CCE, the delivery of its initial operating capability (the date 
when it was to begin providing common e-mail capabilities to ICE users) was delayed from 
September 2005 to December 2005. According to the project manager, this delay is attributable to, 
among other things, the project not following these practices. Specifically, the project manager told us 
that the project experienced numerous changes to requirements when it was transferred from the 
Department of Justice to DHS and that these changes were not documented well. As a result, there 
was extensive system rework to address missing requirements, which caused schedule delays. 
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Observation 2: Key contract management and oversight practices are not fully implemented 

The Software Engineering Institute's CMMI® 22 identifies best practices that are essential to effectively 
managing and overseeing contracts that support IT projects. These best practices include: 

■ establishing and maintaining a plan for managing and overseeing contracts, 

■ assigning responsibility and authority for performing contract management and oversight, 

■ training staff performing contract management and oversight activities, 

■ documenting each contract, including the statement of work and acceptance criteria, 

■ verifying and accepting contractor-provided products and services (i.e., deliverables), and 

■ conducting reviews with contactors to ensure that cost and schedule commitments are being met 
and risks are being managed. 

As we reported 23 in June 2006, the Atlas program has largely established policies and procedures for 
these practices. While the program has implemented two of the practices, it has not fully implemented 
the other four. Specifically, our analysis of the program's performance of these practices, including 
efforts planned and underway, are shown in the following table. 



22 CMMI is registered in the U.S. Patent and Trademark Office by Carnegie Mellon University. 

23 GAO, Homeland Security: Contract Management and Oversight for Visitor and Immigrant Status Program Need to Be Strengthened, 
GAO-06-404 (Washington, D.C.: June 9, 2006). 
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Implementation of Contract Management and Oversight Practices for Atlas Program 




CMMI practice 


Implemented 


GAO's assessment of compliance 


1 


Establish and 
maintain a plan for 
managing and 
overseeing 
contracts. 


no 


The program office issued a plan in February 2006 (referred to as an acquisition plan by 
program officials), but it does not describe contract management and oversight processes. 
Instead, the plan simply states that Atlas is to have a COTR work with program and 
contracting officials to administer the contracts. In November 2006, the DHS Acquisition 
Policy and Oversight Office, which is responsible for reviewing and approving the 
acquisition plan, stated that the plan lacked details on contract management and oversight 
and recommended that Atlas develop, among other things, a contract management and 
oversight plan and incorporate it into the acquisition plan. In addition, we reported in July 
2006 that the ICE System Lifecycle Management Handbook, which is used to help manage 
the Atlas program, does not address certain key acquisition management activities, such 
as contract tracking and oversight. 24 Atlas's program manager stated that the acquisition 
plan and the handbook are being revised to include contract management and oversight 
activities. The acquisition plan is to be revised by March 2007 and the handbook by June 
2007. 


2 


Assign 

responsibility and 
authority for 
performing 
contract 

management and 
oversight. 


yes 


The program office hired a COTR in November 2006, and ICE assigned the individual the 
responsibility and authority for Atlas contract management and oversight. The program 
office also provided one contractor staff member to assist the COTR. The contractor's 
responsibilities include helping develop statements of work, delivery schedules, and 
inspection and acceptance procedures. Further, each of the project managers has been 
tasked by the program manager to assist the COTR in identifying requirements and 
providing assistance in determining whether contractor deliverables meet requirements. 



'GAO-06-823. 
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Objective 2 Results 

Observation 2: Contract Management and Oversight 



Implementation of Contract Management 
CM Ml practice | Implemented 



and Oversight Practices for Atlas Program (cont.) 

GAO's assessment of compliance 



Train staff 
performing 
contract 

management and 

oversight 

activities. 



yes 



The program's COTR has extensive contract oversight training and experience. For 
example, during 2005 and 2006, the official took 120 hours of training on topics such as 
systems acquisition management, COTR responsibilities, the use of government purchase 
cards, and contracting officer's representative mentoring. Further, this official's prior 
experience includes 5 years of managing and overseeing contracts at the Department of 
Defense, at a civilian agency, and in private industry. Also, all but one of the Atlas project 
managers received contract management and oversight training in October 2006. The 
remaining project manager is to receive the training by June 2007. 



Document each 
contract, including 
clearly identifying 
the work to be 
performed and 
acceptance 
criteria. 



no 



On its three key projects — namely CCE, Integration, and the ICE Mission Information 
projects — the program did not satisfy this practice. Specifically, the contracts for these 
three projects did not clearly identify the work to be performed and the criteria for accepting 
contractor deliverables. Instead, the contracts identified the work to be performed and the 
acceptance criteria in general terms. For example, the contract for deploying CCE (a 
standard e-mail system agencywide) states that the contractor is responsible for providing 
deployment services, but does not identify that deploying a standard e-mail system is one 
of those services. Similarly, regarding acceptance criteria, the CCE contract states that the 
contractor will provide monthly status reports, but does not identify how the program will 
determine whether the deliverable is acceptable, such as describing tests to be performed. 



Program officials, including the program manager and the COTR, acknowledged that the 
statement of work and the acceptance criteria are not clearly defined, and that this makes 
contract administration difficult. They added that this difficulty extended to the other Atlas 
contracts because the program managed these contracts in the same manner as the other 
three. To address the problem, the program manager stated that he has tasked the COTR 
to work with project managers in developing clear statements of work and acceptance 
criteria for new contracts that are to be issued by late February 2007 to replace existing 
Atlas contracts that will begin to expire at that time. 
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Implementation of Contract Management and Oversight Practices for Atlas Program (cont.) 




CMMI practice 


Implemented 


GAO's assessment of compliance 


5 


Verify and accept 
the deliverables. 


partially 


The COTR's responsibilities include verifying and accepting contract deliverables. On the 
contracts of the three projects reviewed, the COTR, in assessing whether to accept 
deliverables, did not compare them with specific criteria in the contract because none 
existed (see the previous slide). Instead, the COTR, in consultation with the staff from the 
involved projects and ICE offices of budget and acquisition management, reviewed each 
contractor's invoices to assess whether it appeared what the contractor was charging was 
reasonable for the deliverables provided. If consensus was reached among these parties, 
then the COTR would approve the invoices for payment. 


6 


Conduct reviews 
with contractors to 
ensure cost and 
schedule 
commitments are 
being met and 
risks are 
managed. 


partially 


The program manager stated that he and the ICE CIO conduct reviews with program 
contractors on a quarterly basis to determine whether cost and schedule commitments are 
being met and to review project risks. However, he could not provide documentation that 
the meetings occurred due to the program not documenting the meetings. In addition, the 
contractors provide reports to program officials on their progress in meeting commitments 
and to identify risks related to each project. However, according to the program manager, 
assessing meaningful contractor progress from these reports is difficult without clearly 
defined work-to-be-performed statements and acceptance criteria. 



Source: GAO analysis of DHS data. 
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The Atlas program manager stated that the program did not fully adhere to these practices due to it 
not having until recently a COTR to manage and ensure compliance with contract management and 
oversight requirements. He also stated that to strengthen the program's compliance with such 
practices, he has tasked the COTR with (1) revising the acquisition plan (by March 2007) to provide 
detailed guidance on how Atlas will manage and oversee contracts, (2) revising the ICE System 
Lifecycle Management Handbook by June 2007 to incorporate key contract management activities 
such as contract tracking and oversight, and (3) preparing statements of work (beginning in February 
2007) for future contracts to clearly define Atlas work statements and acceptance criteria. While these 
are steps in the right direction, they are works in progress and, as such, have not been completed. 

Until these steps and the other practices are fully implemented and institutionalized, Atlas faces the 
risk that it will not deliver required capabilities and promised benefits on time and within budget. It also 
makes the program vulnerable to contract mismanagement. 
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Observation 3: Risk Management 

Observation 3: Key risk management practices have yet to be implemented 

Federal guidance and related best practices 25 identify key practices essential for effectively managing 
risks that may impact an IT project. They include, among other things: 

■ identifying and prioritizing risks as to their probability of occurrence and impact, as well as 
documenting them in an inventory; 

■ developing and implementing the appropriate risk mitigation strategies; and 

■ reporting to management on the existence and status of risks and progress in implementing 
mitigation strategies. 

In July 2006, we reported 26 that the program had developed a risk management plan and process to 
guide program office staff in managing risks throughout each project's life cycle. We also reported 
that although the program had recently begun to implement the risk management process, a 
complete inventory of risks had not been developed. Since then, the program has taken additional 
steps to implement risk management. For example, the program hired a risk management coordinator 
to maintain and update the risk management plan, facilitate risk assessments, track efforts to reduce 
risks to acceptable levels, and develop and conduct risk management training. In addition, the 
program has conducted periodic risk management meetings and quarterly program management 
meetings to review the program's progress in managing risks. 



i 

iGAO 

Accountability * Integrity * Reliability 



'See, for example, Software Engineering Institute, Capability Maturity Model Integrated (CMMI), version 1.1 (March 2002). 
5 GAO-06-823. 
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Objective 2 Results 

Observation 3: Risk Management 

Further, the program conducted risk management training with project managers on October 24, 
2006. The training included guidance on, among other things, adhering to key risk management 
practices such as preparing risk assessments and developing risk mitigation strategies. 

However, the program has yet to implement other key practices. First, it does not have a transparent, 
documented, and traceable process for inventorying and resolving risks. For example, the January 
23, 2006, inventory identified three risks related to data/information, security, and privacy. These risks 
were previously reported as having a medium probability of occurrence and a low impact. However, 
the January 23, 2007, inventory did not include these risks. Program officials explained that the 
discrepancy with the number of risks in the inventory is that the program periodically closes risks and 
removes them from the inventory. However, these officials provided documentation on only one risk 
that had been removed from the inventory, and it did not include a clear rationale for why the risk had 
been removed. In addition, the officials were unable to provide documentation showing that the other 
two risks had been closed and removed and the justification for doing so. 

Second, while the risk management plan defines a process and associated practices for developing 
risk mitigation strategies, the program has not fully implemented them. Specifically, the plan specifies 
that mitigation strategies are to include: 

■ a rationale for mitigation strategy chosen; 

■ an explanation of the impact on program performance; 
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Observation 3: Risk Management 

■ a proposed schedule showing milestones for initiation, significant risk mitigation activities, 
and completion; and 

■ the official responsible for implementing and tracking mitigation activities. 
Out of the 61 risk mitigation strategies documented in the January 23, 2007, inventory: 

■ all included a rationale for mitigation strategy chosen; 

■ 13 did not include an explanation of the impact on program performance; 

■ none included a proposed schedule showing milestones for initiation, significant risk mitigation 
activities, and completion; and 

■ all identified officials responsible for implementing and tracking mitigation activities. 

Third, although the risk management plan calls for developing a process for elevating risks to 
management's attention, the program has not developed and implemented such a process. According 
to the program manager, in lieu of a defined escalation process, program officials do make ICE's 
deputy CIO (and sometimes the CIO) aware of program risks as part of quarterly program 
management meetings, but acknowledged that, for the most part, such reviews — and thus the 
escalation of risks — occur in an ad hoc fashion and are not documented. 
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Observation 3: Risk Management 

The program manager stated that the reason for the current state of the program's risk management 
was that the process is new and that it will take time for the risk program to fully mature. He added 
that the program is in the process of addressing these weaknesses by, for example, revising the risk 
management plan to include procedures for (1) reporting to management on the existence and status 
of risks and (2) closing risks. More recently, on January 25, 2007, the program manager provided us 
with the revised plan; it calls for the risk management coordinator to, among other things, establish 
processes for elevating and reporting to management risks that are likely to occur and have a high 
impact. Further, at our February 1 , 2007, exit meeting with program officials, they provided a 
document, approved on January 25, 2007, defining their processes for elevating risk. While these are 
steps in the right direction, the program manager acknowledged that program staff were just 
beginning to implement the processes. 

Until the Atlas program fully implements and institutionalizes risk management, there is increased 
probability that program risks will not be proactively mitigated and, thus, will become actual program 
cost, schedule, and performance shortfalls. 
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Observation 4: Atlas program still implementing performance management practices essential 
to measuring progress against commitments 

As we have previously reported, 27 to enable Atlas program management to measure progress and 
make well-informed investment decisions, it is important for the program to develop and implement 
rigorous performance management practices that include, among other things, properly aligned goals 
and anticipated achievements that are defined in measurable terms. In our September 2005 report, 28 
we recommended that the Atlas program develop and implement an effective performance 
management system that includes such goals and measures. More recently, we reported 29 in July 
2006 that Atlas had begun taking steps to implement performance goals and measures, but had not 
yet completed defining and implementing them. For example, the program had developed goals for 
four of its seven projects but had not done so for its three other projects, which were 

■ Transformation Planning, 

■ Architecture Engineering, and 

■ Data Center Migration. 

We also reported that the program had developed measures to gauge progress on the goals, but had 
not yet begun to track their progress. 
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27 GAO-06-823. 

2S GAO, Information Technology: Management Improvements Needed on Immigration and Customs Enforcement's Infrastructure 

Modernization Program, GAO-05-805 (Washington, D.C.: Sept. 7, 2005). 

29 GAO-06-823. 

41 



Page 48 



GAO-07-565 Information Technology 



Appendix I 

Briefing to the Staffs of the Subcommittees 
on Homeland Security, Senate and House 
Committees on Appropriations 



Objective 2 Results 

Observation 4: Performance Management 

Since then, the program has taken additional steps to define and implement performance goals and 
measures. For example, the program identified 13 goals for five of its seven projects that it has been 
using to measure progress during fiscal year 2006. Twelve of these goals either were expected to be 
finished by fiscal year 2006 or were expected to have some interim measure completed during fiscal 
year 2006. The expenditure plan identified an additional goal (goal 12) that did not have an interim 
measure, but rather was expected to be finished by December 2006. 

In addition, the program now reports it is measuring progress against these performance goals by, for 
example, using them to discuss each project's status and progress at quarterly program management 
reviews. 

The following table provides our analysis of the program's performance goals against its reported 
progress (as of Sept. 30, 2006). 
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Atlas Project Progress in Meeting Fiscal Year 2006 Goals 


Project 


Goal 


Did Atlas 
meet goal? 


If no, what is 
progress against 
goal? 


Overall 

Program 

Management/ 

Transformation 

Planning 


Goal 1 : 1 00 percent of Atlas project activities that are managed with 
less than a 10 percent variance in cost and schedule per fiscal year 


no 


93 percent 


Goal 2: 30 percent reduction in ICE's operations and maintenance 
contribution for network services 


yes 


not applicable 


Goal 3: The enterprise project management tool, Primavera, was 
scheduled for installation, configuration, and deployment in third 
quarter of fiscal year 2006 


no 


deployed in 4 lh quarter 
of FY 2006 


Goal 4: Performance management processes and balanced scorecard 
to be developed and implemented in fourth quarter of fiscal year 2006 


no 


* 


Common 

Computing 

Environment 


Goal 5: 100 percent of ICE personnel using the DHS standard 
Microsoft Outlook E-mail platform 


no 


95 percent 


Goal 6: 33 percent progress in implementing a refresh of ICE 
personnel desktop equipment 


no 


20 percent 


Goal 7: 100 percent of ICE office sites achieving workforce productivity 
upgrades through implementation of ICE e-mail standard, backend 
server upgrades, and ICENet connections 


no 


95 percent 



*While the program did not meet this goal in the fourth quarter of FY 2006 as planned, program officials told us that the program has efforts underway to reach this 
goal, but did not provide a date for when it would be accomplished. 
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Atlas Project Progress in Meeting Fiscal Year 2006 Goals (cont.) 






Did Atlas 


If no, what is 


Project 


Goal 


meet goal? 


progress against 
goal? 


Integration 


Goal 8: 100 percent of ICE sites using DHS standard high-speed 
network circuits 


no 


73 percent 




Goal 9: 100 percent of ICE office sites achieving workforce productivity 
upgrades through implementation of ICE e-mail standard, backend 
server upgrades, and ICENet connections 


no 


73 percent 


ICE Mission 
Information 


Goal 10: 14 percent of investigative and enforcement personnel with 
access to decision support system data marts 


yes 


not applicable 




Goal 11:8 percent increase in ICE investigative and enforcement 
systems incorporated into ICE decision support system consolidated 
data marts 


yes 


not applicable 




Goal 12: The ICE Mission Information project will implement a 
technology model in the first-quarter of FY 2007 for the integration of 
systems and information throughout ICE within the framework of the 
DHS EA. 


Not 
applicable 
(target goal 
set for FY 
2007, but 
not for FY 
2006) 


** 


Information 
Assurance 


Goal 13: 30 percent of ICE users employing single sign-on capability 
for systems access. 


No 


percent 



Source: GAO analysis of DHS data. 



**The program reported that it met this goal in fiscal year 2007, as planned. 
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However, while the program has taken additional steps, it has yet to fully implement and 
institutionalize its performance goals and measures. Specifically, the program has not yet developed 
goals for these two projects — Architecture Engineering and Data Center Migration. According to 
program officials, they have developed draft goals for these projects and are in the process of having 
them reviewed by management, but officials did not provide a date for when the goals are to be 
approved and implemented. 

In addition, the progress on the performance goals, as reported by the program, shows mixed results. 
As shown in the chart, of the 12 goals, the program has met only 3 of them. Of the remaining 9, Atlas 
was close to meeting its fiscal year 2006 targets. For example, on goal 5, the fiscal year 2006 target 
was 100 percent of ICE personnel using the DHS standard Microsoft Outlook e-mail platform, and the 
program reported 95 percent of users have been migrated. Also, on goal 8, the fiscal year 2006 target 
was 100 percent of ICE sites using DHS standard high speed network circuits, and the program 
reported 73 percent of ICE sites are using the circuits. 

Further, underlying data used by the program to measure performance may not be reliable in all 
cases. For example, on goal 1 , the program uses project cost and schedule data from the 2005 Atlas 
business case in measuring progress against the goal, including determining whether variances have 
occurred. However, these data are out of date (i.e., they do not reflect project cost and schedule 
slippages that have occurred since 2005) and are currently being revised as part of a larger effort to 
update the entire business case to reflect overall program changes. 
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In addition, the program relies on its project managers to develop estimates of where their projects 
should be (with respect to schedule) given funds used to date. The program official responsible for 
determining progress against goal 1 told us that this is a key weakness of the performance 
management process because it allows project managers to report on their progress without the 
program having a means to verify and validate their assessments. The program manager 
acknowledged that having and using reliable information to support measurement against goals is a 
challenge across all of the program measures, and the program plans to address the problem by, 
among other things, updating the business case and supporting cost and schedule analyses (which 
are to be issued in September 2007) and by implementing and using an enterprise project 
management tool (planned to start in October 2007) that calculates progress against funds used for 
each project. While these are steps in the right direction, it also means it will be months before the 
program has data it can rely on to measure performance. 

According to the program manager, the program's limited progress in implementing performance 
management practices is attributable in large part to (1) the program's evolving of its performance 
goals as the program evolves; (2) performance management being new to the program, and it will 
take time for the program's processes to fully mature; and (3) the enterprise project management 
tool's taking longer to implement than originally planned. According to the program manager, the 
program recently purchased the tool and plans to begin using it for all projects beginning in October 
2007. Nonetheless, until the program completes implementation and institutionalization of its 
performance management capabilities, including well-defined goals and measures, its managers will 
not have the necessary information for measuring progress and making well-informed investment 
decisions. 
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G A 
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The fiscal year 2006 Atlas expenditure plan, in combination with related program documentation and 
program officials' statements, satisfies or partially satisfies the legislative conditions set forth by 
Congress. However, this satisfaction is based on plans and commitments that provide for meeting 
these conditions rather than on completed actions to satisfy the conditions. In addition, while steps 
are being initiated that are intended to address significant program management weaknesses, a 
number of improvements, including those recommended in our past reports, have yet to be 
implemented. Further, the program has not fully achieved many performance goals that it set out to 
accomplish over the past year. These factors continue to put the program at risk and call for 
heightened and sustained management attention to expeditiously address and resolve the issues. 

Thus, there is much that still needs to be accomplished to minimize the risks associated with the 
program's capacity to deliver promised IT infrastructure capabilities and benefits on time and within 
budget. This includes demonstrating better progress against established performance goals in the 
coming year. Given that hundreds of millions of dollars are to be invested and the program is critical 
to supporting the ICE mission, it is essential that DHS follow through on its commitments to build the 
capacity to effectively manage the program. Proceeding without this capacity introduces unnecessary 
risks to the program and potentially jeopardizes its viability for future investment. 
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To minimize risks to the Atlas program, we recommend that the Secretary of Homeland Security 
direct the Assistant Secretary for Immigration and Customs Enforcement to ensure that ICE follows 
through on its commitments to implement effective management controls and capabilities by 
implementing the following five recommendations: 

■ Employing practices essential to ensuring that the Atlas program's IV&V agent is and remains 
independent, including incorporating requirements in future contracting actions such as the 
renegotiation or recompetition of the current independent verification and validation contract, which 
will prohibit the agent from soliciting, proposing, or being awarded program work other than 
providing independent verification and validation services and products. 

■ Fully adhering to requirements development and management practices, including those specified 
in ICE's policies and procedures. This should also include having the Atlas program manager 
develop a process improvement plan for all of the projects that is consistent with the ICE System 
Lifecycle Management Handbook and provide for making the program manager and project 
managers responsible and accountable for rigorously adhering to the requirements in the 
handbook. 

■ Fully implementing key contract management and oversight practices, including those specified in 
ICE's policies and procedures. This should also include ensuring that the Atlas program manager, 
working with the program's COTR, follow through on planned efforts to strengthen the program's 
compliance with these practices by (1) revising the acquisition plan by March 2007; (2) revising the 
ICE System Lifecycle Management Handbook by June 2007 to incorporate key contract 
management activities such as contract tracking and oversight; and (3) preparing statements of 
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work (beginning in February 2007) for future contracts to clearly define Atlas work statements and 
acceptance criteria. 

Completing implementation of planned risk management activities. This should include (1) fully 
implementing and institutionalizing procedures for reporting to management on the existence and 
status of risks and progress in implementing mitigation strategies and (2) updating the risk 
inventory to include risks for all projects and risk areas. 

Improving program performance management, including developing performance goals for projects 
that do not have goals and reporting on their progress in the fiscal year 2007 expenditure plan. 
Further, the program should assess whether the data being used to measure progress are reliable, 
complete, and accurate. 
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In their February 1 , 2007, oral comments on a draft of this briefing, ICE officials, including the Deputy 
CIO and Atlas program manager, agreed with our findings, conclusions, and recommendations. 
These officials also provided technical comments, which we incorporated in the briefing as 
appropriate. 
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Detailed Scope and Methodology 

To accomplish our objectives, we 

■ analyzed the fiscal year 2006 Atlas expenditure plan and supporting documents against 
legislative conditions and other relevant federal requirements, guidance, and best practices 
to determine whether the conditions were met (in doing so, we considered the conditions 
met when the expenditure plan, including supporting program documentation and program 
officials' representations, either satisfied or provided for satisfying the conditions) and 

■ evaluated supporting documentation and interviewed program and other involved ICE and 
DHS officials to determine progress in establishing capabilities in program management 
areas, such as acquisition planning, enterprise architecture, project management, 
requirements development and management, contract management and oversight, and risk 
management and performance management. 

In assessing the management of requirements development and contract management and oversight, 
we focused on three projects — Common Computing Environment, Integration, and ICE Mission 
Information — that collectively accounted for approximately 75 percent of the funds provided to the 
program and the Atlas program manager identified as being the most critical to the program's success 
and ICE's mission. For DHS and ICE data that we did not substantiate, we made appropriate 
attribution indicating the data source. We conducted our work at ICE and DHS headquarters in 
Washington, D.C., from November 2006 through January 2007 in accordance with generally accepted 
government auditing standards. 



G A O 

Accountability * Integrity * Reliabilitj 
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ICE Requirements Management Efforts Compared with Federal IT Management Guidance and 
Best Practices 


Establish a requirements management policy 


Project 


Implemented 


GAO's assessment of compliance 


CCE 


yes 


In February 2005, the CCE project established the ICE System Lifecycle Management 
Handbook? as the policy to guide project efforts in developing and managing requirements. The 
handbook requires requirements development and management activities, such as eliciting 
desired system capabilities from users and translating them into requirements and obtaining 
approval by all stakeholders. 


Integration 


yes 


In February 2005, the Integration project established the ICE System Lifecycle Management 
Handbook as the policy to guide project efforts in developing and managing requirements. 


ICE Mission 
Information 


yes 


In February 2005, the ICE Mission Information project established the ICE System Lifecycle 
Management Handbook as the policy to guide project efforts in developing and managing 
requirements. 



Source: GAO analysis of DHS data. 

Legend: Yes=established/implemented; No=not established/implemented; Partially=some, but not all, actions are implemented or actions are in progress of being 
implemented 



U.S. Immigration and Customs Enforcement: System Lifecycle Management, The Systems Development Lifecycle of Immigration and 
Customs Enforcement, version 1.0. (Washington, D.C.: February 2005). 
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ICE Requirements Management Efforts Compared with Federal IT Management Guidance and Best 
Practices (cont.) 


Elicit desired or required system capabilities from users and translate them into system requirements and 
obtain approval by all stakeholders 


Project 


Implemented 


GAO's assessment of compliance 


Common 

Computing 

Environment 


partially 


In 2003, the CCE project elicited system capabilities from component and departmental users. 
Although the capabilities elicited from the users were not documented, project officials told us 
they translated the users' input into system requirements, issuing a draft requirements 
document in June 2004. Although the draft requirements had not yet been reviewed and 
approved by system stakeholders, the project began deploying system capabilities in December 
2005. Project officials told us that they planned to finalize the requirements document, including 
obtaining stakeholder approval, and in January 2007, they provided us with a finalized and 
approved version of the requirements document. 


Integration 


partially 


In 2003, Integration elicited system capabilities based on a directive from the DHS Deputy 
Secretary and departmental users. The capabilities were elicited from users at requirements 
workshops and documented. Project officials told us that they translated the users' input into 
system requirements, issuing a draft requirements document on June 15, 2005. Although the 
draft requirements have not yet been reviewed and approved by system stakeholders, the 
project began deploying system capabilities in March 2006. Project officials told us that they 
have obtained stakeholder approval on the requirements document; however, they could not 
provide documentation showing this had occurred. 


ICE Mission 
Information 


partially 


Divided into four phases, the ICE Mission Information project is currently in the process of 
eliciting user system capabilities for each of the respective phases. For the first phase, the 
project has elicited system capabilities from its users, but has not documented them. Project 
officials told us that they translated the user capabilities into system requirements, issuing a 
draft requirements document dated August 7, 2006. Project officials stated that they planned to 
finalize the requirements document, including obtaining system stakeholder approval, but could 
not provide a date by when this was to occur. For the other three phases, the project has not 
completed eliciting requirements and documenting capabilities, but project officials told us they 
planned to do so in accordance with the practices. However, they were not able to provide a 
plan and associated timetable for when they are to complete the practices. 
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ICE Requirements Management Efforts Compared with Federal IT Management Guidance and Best 
Practices (cont.) 


Develop a system concept of operations 


Project 


Implemented 


GAO's assessment of compliance 


CCE 


partially 


CCE did not develop a concept of operations, which is integral to defining system requirements, 
until 3 years after the project had been initiated. Specifically, the project was initiated in March 
2003 and began deploying system capabilities in December 2005, and the concept of 
operations was developed in June 2006. 


Integration 


no 


According to the Integration project manager, a concept of operations was developed. However, 
Integration was not able to provide documentation showing it had been developed. 


ICE Mission 
Information 


partially 


The project issued a draft concept of operations on October 12, 2006. The ICE Mission 
Information project manager told us that the project is in the process of finalizing the document 
but did not provide a date for when the document would be completed. 



Source: GAO analysis of DHS data. 
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ICE Requirements Management Efforts Compared with Federal IT Management Guidance and Best 
Practices (cont.) 


Ensure that changes to requirements are controlled and approved via a disciplined change control process 
as the system is developed and implemented 


Project 


Implemented 


GAO's assessment of compliance 


CCE 


partially 


CCE did not follow a disciplined change management process for 3 years after the project was 
initiated in March 2003; during this time, requirements were introduced and implemented in an 
ad hoc fashion. The program has taken steps to improve change management efforts by (1) 
developing an Atlas Configuration Management Plan on February 15, 2006; (2) developing a 
CCE change control board charter on November 1, 2006; and (3) developing a change process 
that provides the procedures for reviewing and approving CCE change requests. The project is 
currently in the process of implementing these changes. Project officials told us that the project 
recently began conducting CCE change control board meetings in September 2006. According 
to the change process, CCE is required to ensure that technical, cost, and schedule impacts are 
considered before approval is granted. Although program documentation showed the technical 
impact of the change, the cost and schedule impacts were not documented. 


Integration 


partially 


The configuration control board and change control process that Integration follows was 
developed by DHS's Infrastructure Transformation Program, which is the departmental 
organization (commonly referred to as the "network steward" by DHS and ICE officials) 
responsible for ensuring that all DHS components migrate from their existing networks to the 
DHS network. However, Integration officials could not provide evidence of compliance with the 
network steward's change control process. 


ICE Mission 
Information 


partially 


The project plans to control and approve changes via a change control process when the 
requirements are developed and finalized. The project plans to do so consistent with the 
guidance specified in the ICE System Lifecycle Management Handbook. As previously noted, 
project officials were not able to identify when the requirements would be finalized and 
approved. 
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Attachment II 

GAO Analysis of Requirements Management 



ICE Requirements Management Efforts Compared with Federal IT Management Guidance and Best 
Practices (cont.) 


Maintain bidirectional traceability, meaning that a given requirement can be traced backward to systems 
documentation and forward to the appropriate contract vehicle and system components 


Project 


Implemented 


GAO's assessment of compliance 


CCE 


partially 


In December 2006, CCE developed a requirements traceability analysis document — commonly 
referred to as a matrix — to show bidirectional traceability between the requirements and system 
documentation. While the project identified requirements, it did not trace requirements to the 
system documentation, such as the system design and test plans. Additionally, the project did 
not trace requirements to the existing contract task order. 


Integration 


partially 


In June 2005, Integration developed a requirements traceability analysis document — commonly 
referred to as a matrix — to show bidirectional traceability between the requirements and system 
documentation. While the project identified requirements, it did not trace requirements to the 
system documentation, such as the system design and test plans. Additionally, the project did 
not trace requirements to the existing contract task order. 


ICE Mission 
Information 


partially 


The project plans to ensure traceability among requirements, system documentation, and the 
existing contract task order when the requirements are developed and finalized. The project 
intends to do so in accordance with the guidance specified in the ICE System Lifecycle 
Management Handbook. As previously noted, project officials were not able to identify when the 
requirements would be finalized and approved. 



Source: GAO analysis of DHS data. 
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U.S. Department of Homeland Security 

Washington, DC 20528 



?Sf Homeland 
■^'^ W Security 



April 5, 2007 



Mr. Randolph C. Hite 
Director 

Information Technology Architecture and 
Systems Issues 

U.S. Government Accountability Office 
441 G Street, NW 
Washington, DC 20548 

Dear Mr. Hite: 

Thank you for the opportunity to review and comment on the Government Accountability 
Office's (GAO's) draft report GAO 07-565 entitled INFORMATION TECHNOLOGY: 
Immigration and Customs Enforcement Needs to Fully Address Significant Infrastructure 
Modernization Program Management Weaknesses. 

U.S. Immigration and Customs Enforcement (ICE) concurs with the draft report's five 
recommendations. The following describes the actions ICE has taken or plans to take to 
implement the recommendations. 

Recommendation 1: Employ practices essential to assuring that the Atlas program's 
independent verification and validation agent is and remains independent, including 
incorporating requirements in future contracting actions such as the renegotiation or 
recompetition of the current independent verification and validation contract, which will 
prohibit the agent from soliciting, proposing, or being awarded program work other than 
providing independent verification and validation services and products. 

Response: Future Atlas contract actions will prohibit independent verification and 
validation (IV&V) agents from soliciting, proposing, or being awarded non-IV&V Atlas 
work to prevent a conflict of interest situation. 

Recommendation 2: Fully adhere to requirements development and management 
practices, including those specified in ICE's policies and procedures. This should also 
include having the Atlas program manager develop a process improvement plan for all 
the projects that is consistent with the ICE System Lifecycle Management Handbook and 
provide for making the program manager and project managers responsible and 
accountable for rigorously adhering to the requirements in the handbook. 
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Response: The Atlas processes are being refined to fully comply with Systems Lifecycle 
Management standards. By May 31, 2007, ICE anticipates the implementation of an 
improvement plan that will address requirement development and management processes, 
training, and the establishment of a Change Control Board to govern and manage 
impacts. 

Recommendation 3: Fully implement key contract management and oversight practices, 
including those specified in ICE's policies and procedures. This should also include 
ensuring that the Atlas program manager, working with the program's contracting 
officer's technical representative, follow through on planned efforts to strengthen the 
program's compliance with these practices by (1) revising the acquisition plan by May 
2007, (2) revising the ICE System Lifecycle Management Handbook by June 2007 to 
incorporate key contract management activities such as contract tracking and oversight, 
and (3) preparing statements of work (beginning in February 2007) for future contracts to 
clearly define Atlas work statements and acceptance criteria. 

Response: In November 2006, a full-time contracting officer's technical representative 
joined the Atlas program to implement industry compliant contract management and 
oversight policies. Furthermore, the Atlas Acquisition Plan is currently being revised and 
is proceeding ahead of the May 2007 schedule. The Atlas program will make the newly 
instituted contract tracking and management oversight processes available for 
incorporation into the next revision of the ICE System Lifecycle Management Handbook 
scheduled for release in June 2007. The use of a standardized statement of work (SOW) 
template with sections for detailed requirements and acceptance criteria has been 
instituted for the creation of all Atlas SOWs. 

Recommendation 4: Complete implementation of planned risk management activities. 
This should include (1) updating the risk inventory to include risks for all projects and 
risk areas and (2) fully implementing and institutionalizing procedures for reporting to 
management on the existence and status of risks and progress in implementing mitigation 
strategies. 

Response: The Atlas risk management and escalation processes have been refined to 
ensure effective risk management and better communication of risk, and were deployed 
in March 2007. Furthermore, the risk inventory has been updated to re-incorporate 
all risks previously archived due to their closed status, to report risk as it relates to impact 
on cost, schedule and performance, to detail risk mitigation strategy, and track status of 
risks and progress. 

Recommendation 5: Improve program performance management, including developing 
performance goals for projects that do not have goals and reporting on their progress in 
the fiscal year 2007 expenditure plan. Further, the program should assess that the data 
being used to measure progress are reliable, complete, and accurate. 

Response: Performance goals have been developed for all Atlas projects. Standard 
procedures to include quarterly assessments have been deployed and are currently 
employed to ensure measurement accuracy. 
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Thank you again for the opportunity to comment on this draft report and we look forward 
to working with you on future homeland security issues. 

Sincerely, 

Steven J. Pecinovsky 
Director 

Departmental GAO/OIG Liaison Office 
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The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting its 
constitutional responsibilities and to help improve the performance and 
accountability of the federal government for the American people. GAO 
examines the use of public funds; evaluates federal programs and policies; 
and provides analyses, recommendations, and other assistance to help 
Congress make informed oversight, policy, and funding decisions. GAO's 
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accountability, integrity, and reliability. 
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